1. Field of the Invention
The present invention relates to the non-contact identification and inspection of color patterns on a stream of material (e.g. carpeting, fabric, wall paper, printed matter, or a series of discrete objects) traveling along a production line.
2. Statement of the Problem
A need exists for a low cost, high speed system for identifying and inspecting color patterns on materials traveling through a production line. A first field of use involves inspection of a series of discrete objects, such as labels such as those found on beverage cans, in an assembly line environment. The system should adjust to the label configuration so that it can automatically learn the overall color signatures of a label and once the learning process is accomplished, the system should automatically adapt to inspect all subsequent labels. The system should be able to function independently of the orientation of the label and of production line speed, yet capable of operating at high speeds such as 2000 cans per minute. The system must not physically contact the label or interfere with the flow of the containers on the production line. Finally, the system should be capable of inspecting the label for fine defects such as grease spots and scratches on the order of one square centimeter, small changes in color wavelength and intensity, and changes in color balance due to ink smears.
A second field of use involves inspection of a continuous stream of material (e.g. carpeting, fabric, wall paper, or printed matter) moving along a production line. Again, the system should be capable of automatically learning the color signatures of the material stream. The system should be able to function independently of the production line speed, and yet be capable of operating at high speeds. The system must not physically contact with the stream of material or interfere with the flow of material on the production line. Finally, the system should be capable of inspecting the entire width of the stream of material for fine defects, small changes in color wavelength and intensity, and changes in color balance.
3. Results of Patentability Search
A number of optical inspections have been devised in the past, including the following:
______________________________________ INVENTOR U.S. PAT. NO. ISSUE DATE ______________________________________ Fickenscher et al. 3,676,645 7-11-72 Yoshimura et al. 3,745,527 7-10-73 Trogdon 4,270,863 6-2-81 Christian et al. 4,589,141 5-13-86 Dennis 4,790,022 12-6-88 Tajima 4,797,937 1-10-89 Kappner 4,809,342 2-28-89 Schrader et al. 4,859,863 8-22-89 Uchida et al. 4,881,268 11-14-89 ______________________________________
Cutler-Hammer Product Information (Eaton).
Multiple LED Color Sensing, by Gregory L. Nadolski.
The 1989 patent to Uchida pertains to a system using optical fiber bundles disposed so as to identify a particular type of bank note by detecting colors from reflected or transmitted light. The Uchida approach utilized three color detecting sensors to receive reflective light from a selected linear path on the bank notes being inspected. Hence, Uchida is limited in that it does not perform a complete label inspection, but rather only narrow linear portions of the bank note. Hence, defects occurring in other portions of the bank note not in a linear path of one of the detectors would remain undetected. Furthermore, the Uchida sensor utilizes optical fiber bundles which must be located in close proximity to the surface of the bank note. As the bank note moves, a time varying signal is generated. The signal variation repeats for each bank note and, therefore, is cyclical. The time varying signals received by the sensors are processed by hardware into two color components (e.g., blue/red) and the ratio of these components (i.e., red/blue) is obtained. The resulting ratio signal is then compared with a predetermined reference pattern signal which is stored in memory. The bank notes must be precisely oriented in delivery due to the narrow color region being examined. The Uchida system is incapable of self learning and must be provided with the referenced pattern.
The 1988 patent to Dennis sets forth the use of a color camera which produces gamma-corrected RGB output that is fed to three picture stores for green, red, and blue components. This output is delivered through analog to digital converters into a microprocessor. The signal output, like Uchida, is time varying but it is not cyclical since the vegetables are randomly provided. The Dennis approach is suitable for analyzing color differences in vegetables moving along the conveyor line (such as green spots in potatoes). As such, the vegetables can be oriented in any direction and they can be of differing sizes and shapes. Dennis looks for a particular color pattern of perhaps a size and shape that renders the vegetable defective. The system must be first calibrated by utilizing an actual potato containing a defect having an undesirable shade of green and the system is then capable of detecting the transition between the green defect area and the color of the surrounding potato. Dennis detects only a transition defect in a color specific background by using two or three dimensional color patterns stored in a three dimensional memory (which is implemented in three separate two-dimensional look-up tables). This approach is unsuitable for detecting small defects in labels.
Yoshimura provides for precisely oriented postage stamps being delivered through a scanner. Again, this approach is not suitable for randomly oriented containers such as beverage cans in an assembly line. However, Yoshimura only utilizes the three reflected colors: red, green, and blue to address a look-up table to assign a region a color (i.e. red, green, blue or white) based upon the combination of the three inputs. These signals are time varying and are precisely based upon the known geometries of the stamp's design.
The 1989 patent to Schrader is a label inspection apparatus which senses overall reflectivity values of labels moving in a conveyor line at conveyor speeds of 100 to 600 containers per minute with containers spaced at three inch clearances. Labels up to six inches can be read. The invention uses a linear array of photo detectors arranged at 1/2 inch centers on a vertical line. A microprocessor is used to calculate the percentage reflectivity values and pass or fail limits are established for the containers. The invention also includes a learn cycle wherein a sufficiently large statistical sample of containers are read to determine the overall reflectivity values which will represent the entire population of containers to be inspected.
The 1989 patent to Kappner sets forth a process for identifying and recognizing objects such as permanent coding. This invention is able to identify a precise coordinate position for the coded symbols on the object.
The 1989 patent to Tajima pertains to an apparatus for identifying postage stamps. This invention scans postage stamps and detects the various colors contained thereon and which are located at predetermined regions on the stamp. The received color signals are used to produce a feature vector which represents the color distribution over the scanned area. Sensor arrays are used to produce red, green, and blue color analog electrical signals which are digitized based upon color moments within a defined area. The sensor arrays are designed to provide a scanning line and the stamps must be precisely delivered to insure the scanning line integrity.
The 1986 patent to Christian pertains to a computer vision apparatus for automatically inspecting printed labels. This system first goes through a teach phase in which the label is memorized by the system. Secondly, it goes through an inspection phase in which unknown labels are then inspected.
The 1981 patent to Trogdon (U.S. Pat. No. 4,270,863) and assigned to Owen-Illinois Inc. sets forth an apparatus for illuminating the surface of the label and then generating an intensity level for a number of points on the surface of the label which are sensed by a photo sensitive diode array. The intensity levels are then compared with a stored maximum value and if different from that value, a good or bad signal is generated. This invention utilizes a learning process by inspecting a number of labels, storing that information, and then using the stored information to do the inspection. This invention utilizes a camera having a 128 by 128 array. An A/D converter receives the camera analog video signals to generate a digitized signal.
The 1972 patent to Fickenscher sets forth a label reader using a rotating faceted mirror.
Nadolski discusses the use of red, green, and blue LED light sources for on-line color sensing systems.
The Cutler-Hammer literature discusses a color sensing system in which the sensor head is triggered to take a number of color samples. The average color profile is then computed from the samples and compared against a prerecorded standard. The standard values can be "learned" from previously stored samples. This reference does not involve development of color signatures other than average values, and therefore cannot be applied to testing color patterns.
It is believed that Uchida et al., Dennis, and Yoshimura, et al. are the most pertinent to the teachings of the present invention. However, Uchida, et al., require precision in the delivery of each stamp to the three narrow line scanners. Dennis, Uchida, et al. and Yoshimura, et al. require that the system must be initialized with reference values. None of these approaches are designed to sample the entire surface of the object or width of the material stream by first automatically learning the color signatures and then finely inspecting for color defects.
The Uchida, Yoshimura and Dennis patents each store time varying signals and then process those signals to generate color differences vs. position. The generated color signals are compared to the stored color values. A need still exists for a system to obtain single color samples for the stream of material (or objects) being tested and to accumulate such samples to obtain an overall spatial color signature of the stream of material (or objects) which is insensitive to scan rate and which utilizes simple hardware and minimal memory.
4. Solution to the Problem
The present invention offers a solution to the above problem by providing a low cost, high speed system for identifying and inspecting either a series of discrete objects or a two-dimension stream of material. The present invention is capable of performing process defect inspection independent of the material flow rate, yet operates at high rates of speed. The system of the present invention first samples the passing material or objects in order to learn and to construct the color signatures for the entire object or the width of the material stream, as appropriate. When satisfied that learning is completed, the system then automatically configures to inspection mode.
The present invention can have its sensitivity selectively adjusted with maximum sensitivity occurring in twenty-eight different color dimensions coupled with minimum data dilation. Furthermore, the orientation of objects in the stream of material can be random as they pass the optical head and the system of the present invention is still capable of learning the color signatures and performing inspection of the objects. The optical inspection system of the present invention does not physically contact with the material stream or interfere in any fashion with the flow of the production line except to provide a reject signal if desired. The system of the present invention is capable of inspecting for defects such as grease spots and scratches on the order of one square centimeter, small changes in color wavelength and intensity, and changes in color balance due to ink smears.
One overriding difference exists between the Uchida, Yoshimura and Dennis approaches and the approach of the present invention. This difference is in the method that the signature is collected. All three of the prior art machines gather a set of time varying signals produced by moving the object in front of the sensor or by scanning the sensor field of view across the object. What is collected in each case is a signature that contains information regarding the spatial color characteristics of the label on the object. These signals are then processed to generate spatial difference signals.
In contrast, the present invention periodically samples the color intensities of the portion of the stream of material within the field of view of the sensor. Each individual sample typically encompasses only a fragmentary portion of the overall color pattern on the stream of material being inspected. In the case where a series of discrete objects are being inspected, each object passing the sensor can display different data (i.e., either different parts of an object or the same object in different orientations or a combination of both). In either case, the present system uses information collected from a large number of these samples to eventually generate a complete color signature set for the entire color pattern. The information collected on each single pass is different due to the object's spatial orientation, but that spatial orientation is not incorporated into the learned data. Thus, the signature learned by the present invention can be either a function of the varying characteristics of the object along its length or of the varying orientation of the object with respect to the sensor. Since a spatial difference signal is not generated by the present invention, the scan rate, speed of the production line, and delay characteristics do not affect performance. The present invention operates at any line speed from full stop to the maximum rate.
The hardware and software requirements of having to store the data from an entire object scan are eliminated with the present invention since it only collects single samples from each data channel every pass. In the case of the Dennis invention, a significant hardware savings is realized in eliminating the color TV camera, video frame buffers, and associated control circuitry. In the Dennis and Yoshimura inventions, a significant amount of hardware is dedicated to the delay and add functions not required by the present invention.
A second fundamental difference between the present invention and the other prior art approaches set forth above involves the manner in which the present invention produces the color separated signals. The other systems utilize filters over the sensors, or a color TV camera. The present invention passes the reflected light from the sample image through a transmissive diffraction grating to separate the component colors. Any diffractive element or a prism could be utilized for this task. This portion of the machine is inexpensive compared to the costs and complexity of all the other systems. Hence, simplicity is achieved through the use of the grating or prism (i.e., there is only one set of optics for each sensor head).
In comparison to Yoshimura, which is insensitive to irregularities in the object surface and to letter ornamentation and patterns (column 2, line 42), the present invention specifically detects these irregularities. The three-dimensional mapping referred to in Yoshimura is used to characterize a spatial region as red, blue, green, or white based on the RGB inputs from the sensor. This determination is then used to generate appropriate color specific timing signals. The present invention uses the color signals to access a multidimensional memory wherein data is written to perform the learn process or from which data is read to perform the compare process. Thus, the functions of the multidimensional mappings of the present invention are different from Yoshimura. Yoshimura relies heavily on the known and fixed characteristics of the label under test, specifically the relation between the edges and the color borders of the stamps. The present invention assumes no foreknowledge of the stream of material under test and sets no requirements on its characteristics beyond being located within the optical field of view. Yoshimura requires exact placement of the label with respect to the sensor so that a particular region of the label can be compared with the fixed signatures. The present invention is capable of learning object characteristics in any orientation or combination of orientations and aspects of the objects or stream of material. All of the scanned, time varying signals of Yoshimura are further processed by delaying the signal and subtracting it from its original real time signal to create a temporal, and thus a spatial, color difference signal. This signal is then used to generate color-dependent timing signals which create an evaluation metric. Thus, it is the scanned characteristic of Yoshimura which allows it to function. Additionally, the operation of Yoshimura is in part dependent on the scan rate and delay function. The present invention is insensitive to object rate.
In comparison to Uchida which collects a time varying signal from two color sensors and after providing a ratio of the color signals compares them to the stored signature data, the present invention requires no predetermined signatures to make its evaluation. Uchida requires exact placement of the test label with respect to the sensor so that a particular region of the label can be repeatedly compared with the fixed signatures. The present invention is capable of learning object characteristics in any orientation or combination of orientations and aspects of the objects under inspection.
In comparison to Dennis, which uses multiple two-dimensional multi-bit tables and logically AND's their outputs to generate an overall evaluation, the present invention uses only multiple two-dimensional one-bit tables. This results in a savings of computer memory by allowing for the digitization of the color signals into greater numbers of bits than would be practical if multi-bit look-up tables were utilized. Dennis must teach his machine the specific defect to be detected by actually showing the system a sample defect or an image of the sample defect. Furthermore, the sample defect must be seen by the machine against the specific object background on which it can occur (green spot on yellow background, for example). It is this defect signature data that is stored in the look-up tables of the Dennis machine. The present invention is taught what "good" objects or materials look like and it detects any deviation from that learned set. Thus any defect may occur on any portion of the object or material stream without regard to the surrounding characteristics or defect type. Dennis also relies on scanning the object and, like Yoshimura, creates a spatial color difference signal. These difference signals are then used to access the multiple two-dimensional look-up tables to determine if a defect has been detected. Thus, the actual information that is being stored in the tables is different from that stored in the present invention. Dennis stores spatial color difference signals thereby keying off the color transition at the boundary between a good region and a defective region of the object. The present invention stores the actual color intensities from the portion of the object viewed in memory and keys off any deviation from the learned data.
An important capability of the present invention is its ability to learn object characteristics which vary either because of the object's orientation with respect to the sensor or due to the portion of the object viewed by the sensor. In the can inspection application, the random orientation of the cans is exploited to allow the present invention to learn the characteristics of all aspects of a can label. This is not necessary though. If the cans always passed the sensor showing the same portion of the label, the present invention would simply learn that much of the complete signature and would not perform less satisfactorily since subsequent cans would also present only that same portion of the label for inspection. Defects such as color hue shift, misregistration, etc. could still be detected. Of course, if a physical defect always occurred on the opposite side of the can, it would never be detected, but the same would be true of any of the above discussed approaches.